In this paper, a MIMO simulated annealing SA based Q learning method is proposed to control a line follower robot. The conventional controller for these types of robots is the proportional P controller. Considering the unknown mechanical characteristics of the robot and uncertainties such as friction and slippery surfaces, system modeling and controller designing can be extremely challenging. The mathematical modeling for the robot is presented in this paper, and a simulator is designed based on this model. The basic Q learning methods are based pure exploitation and the epsilon-greedy methods, which help exploration, can harm the controller performance after learning completion by exploring nonoptimal actions. The simulated annealing based Q learning method tackles this drawback by decreasing the exploration rate when the learning increases. The simulation and experimental results are provided to evaluate the effectiveness of the proposed controller.

This tutorial was developed upon my last project: Line Follower Robot - PID Control - Android Setup. Once you have a robot with line following capabilities, the next natural step is to give him some degree of intelligence. So, our dear "Rex, the Robot" will try now finding how to scape from a "labyrinth" on a shortest and fastest way (by the way, he hates the Minotaurus. For starting, what is the difference between Maze and Labyrinth? According to http://www.labyrinthos.net, in the English-speaking world it is often considered that to be qualified as a maze, a design must have choices in the pathway.